Combination strategy for prognostication in patients undergoing post-resuscitation care after cardiac arrest

This study investigated the prognostic performance of combination strategies using a multimodal approach in patients treated after cardiac arrest. Prospectively collected registry data were used for this retrospective analysis. Poor outcome was defined as a cerebral performance category of 3–5 at 6 months. Predictors of poor outcome were absence of ocular reflexes (PR/CR) without confounding factors, a highly malignant pattern on the most recent electroencephalography, defined as suppressed background with or without periodic discharges and burst-suppression, high neuron-specific enolase (NSE) after 48 h, and diffuse injury on imaging studies (computed tomography or diffusion-weighted imaging [DWI]) at 72–96 h. The prognostic performances for poor outcomes were analyzed for sensitivity and specificity. A total of 130 patients were included in the analysis. Of these, 68 (52.3%) patients had poor outcomes. The best prognostic performance was observed with the combination of absent PR/CR, high NSE, and diffuse injury on DWI [91.2%, 95% confidence interval (CI) 80.7–97.1], whereas the combination strategy of all available predictors did not improve prognostic performance (87.8%, 95% CI 73.8–95.9). Combining three of the predictors may improve prognostic performance and be more efficient than adding all tests indiscriminately, given limited medical resources.


Post-cardiac arrest care
All patients received standard intensive care according to our institutional intensive care unit protocol based on the 2021 international guidelines for post-CA care 5 .All included patients underwent post-CA care, including targeted temperature management (TTM).TTM was performed using cooling devices (Arctic Sun ® 5000; BD, Franklin Lakes, NJ, USA).The target temperature was determined by the attending physician (33 vs. 36 ℃) according to hemodynamic status or CA characteristics, and then maintained for 24 h with rewarming to 37 °C at a rate of 0.25 ℃ per hour and monitored using an esophageal or bladder temperature probe.Midazolam (0.05 mg/kg intravenous bolus, followed by a titrated intravenous continuous infusion at a dose between 0.05 and 0.2 mg/kg/h) and paralytics (cisatracurium or rocuronium) were administered for sedation and to control shivering.If there was evidence of electrographic seizure or a clinical diagnosis of seizure, antiepileptic drugs, such as levetiracetam and/or valproate, were administered.The sedation level for all patients was assessed using a clinical sedation and agitation score [Richmond Agitation-Sedation Scale (RASS)].Deep sedation, defined as a RASS score of 4 or 5, was maintained for the first 72 h after ROSC.
Since 2018, WLST has been authorized only under high restrictions in Korea, even if the family has a strong willingness to pursue WLST 14 .Legally, declarations of irreversible and unrecoverable status must be obtained from at least two physicians.Therefore, the physicians in charge of post-CA care do not encourage WLST, and it is restrictively performed in patients with brain death who are denied organ donation by the caregiver or family.In addition, Korea accepts a highly strict qualification for brain death, such as a flat activity (< 2 µV) in an electroencephalogram (EEG) for 30 min 15 .

Baseline characteristics
These variables were extracted from the data registry: age, sex, Charlson comorbidity index, sequential organ failure assessment score within the first 24 h after admission, witnessed collapse, bystander cardiopulmonary resuscitation (CPR), time from collapse to CPR (no flow time), time from CPR to the return of spontaneous circulation (ROSC; low flow time), first monitored rhythm, etiology of cardiac arrest, Glasgow Coma Scale (GCS) immediately after ROSC, time to the targeted temperature, and times to perform each prognostic test from ROSC.The following prognostic test data were extracted from the prospectively collected registry and electronic medical record system in our institution, Chungnam National University Hospital, Daejeon, Korea.

Ocular reflexes
Ocular reflexes included pupillary light and corneal reflexes (PR/CR), and an absent finding was defined that they were not observed bilaterally.Data for PR/CR were extracted between 72 and 96 h after ROSC and at the end of sedation to exclude confounding effects of sedative drugs.PR/CR measurements were performed by experienced nurses trained in general critical care, using a manual flash lamp and gauze.

Electroencephalography
EEG was performed for clinical indications at the discretion of the treating physicians in patients whose GCS motor score was below 6.The recordings were performed with the standard international 10-20 system of electrode placement of 21 electrodes for typically 15-30 min (Compumedics E-series, Compumedics, Melbourne, Australia).The EEG records were interpreted based on the recently updated guideline for critical patients by the American Clinical Neurophysiology Society 16 and then qualitatively classified into one of three grades (benign, malignant, or highly malignant patterns) by two neurophysiologists (EYK and JMK) blinded to clinical course and outcome.Suppressed background, suppressed background with continuous periodic discharges, and burstsuppression background were defined as highly malignant patterns and used as poor outcome predictors.A consensus was made in the event of different interpretations of the EEGs between the two experts.Given the current guidelines and rarely performed WLST setting in this study, EEG recordings on the study population obtained between 24 h and 7 days from ROSC were identified through an EEG database.If more than one EEG was obtained, the last EEG of each patient was used for analysis.
www.nature.com/scientificreports/Neuron-specific enolase A peak value of serum neuron-specific enolase (NSE) measured at 48 or 72 h from ROSC was used for the analyses.All samples were obtained from an arterial line.Serum NSE levels were determined using an electrochemiluminescence immunoassay with Elecsys NSE ® (COBAS e801; Roche Diagnostics, Rotkreuz, Switzerland) in an authorized laboratory (GC Labs; Yongin, Geonggi-do, Korea).

Imaging studies
Brain CT and MRI, including diffusion-weighted imaging (DWI) performed after targeted temperature management (TTM) (72 to 96 h from ROSC), were used for this analysis.A gray-white matter ratio at basal ganglia level (GWR-BG) on CT and diffuse high-signal intensity (HSI) on DWI were calculated and estimated, respectively.Our protocol for the interpretation of imaging studies is described in the supplementary material (Supplementary Method and Supplementary Fig. S1).

Outcome
Neurological outcomes were assessed at 6 months after OHCA using the Cerebral Performance Category (CPC) score (see Supplementary Note).The primary outcome in this study was a poor neurological outcome defined as CPC score of 3 to 5.

Statistical analysis
Given that the individual numbers of each combination strategy are expected in this study, we chose a combination strategy with the smallest numbers as a more conservative estimate for sample size.Sample size was estimated using the method described by Buderer 17 .Based on a previous study using a Korean multi-center registry of post-cardiac arrest patients 18 , assuming that the combination strategy using the EEG and CT has a sensitivity of 30% and specificity of 100% for predicting poor neurological outcome in our cohort, a confidence interval (CI) of 20%, and a 80% prevalence of poor neurological outcome in a patients with post-cardiac arrest care, we calculated a sample size of at least 43 patients.
Categorical variables are presented as numbers with percentiles, and continuous variables are presented as means with standard deviation or median values with interquartile range, depending on the normality of the data.Categorical variables were compared between the groups using the Chi-square or Fisher's exact test, as appropriate.Continuous variables were compared between groups using the Student's t-test or the Mann-Whitney U-test.Receiver operating characteristic (ROC) curves were generated for each predictor, and a combined model was generated using logistic regression analysis.The predictive accuracy was determined by the area under the ROC curve (AUC), sensitivities, and false-positive rate (FPR) with 95% confidence interval (CI).Subsequently, the sensitivity was categorized and defined as excellent (> 90%), moderate (70 to 90%), or poor (< 70%) 19 .Inter-rater reliabilities were determined in the interpretations of EEG, CT, and DWI using Cohen's kappa (κ) value and the intraclass correlation coefficient (ICC) according to the characteristics of the variables.The κ value was defined as slight (0.01 to 0.2), fair (0.21 to 0.40), moderate (0.41 to 0.60), substantial (0.61 to 0.80), and almost perfect (0.81 to 1.00) agreement 20 .An ICC value of < 0.4, 0.4-0.75, and > 0.75 indicated poor, fair to good, and excellent agreement, respectively 21 .Bootstrap internal validation was subsequently performed to verify the sensitivity and specificity for poor neurological outcome in each combination strategy.DeLong's test was used to compare AUCs computed without multiple imputations.Statistical analysis was performed using IBM-SPSS 26.0 for Windows (IBM Corp., Armonk, NY, USA) and MedCalc 22.014 (MedCalc Software, Mariakerke, Belgium).The significance level was set at P < 0.05.

Statement of ethics
The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Institutional Review Board (or Ethics Committee) of Chungnam National University Hospital (No. 2021-11-024).The extracted data included clinical data only; it includes no personally identifiable information.Therefore, the need for informed consent was waived.

Study population
Of the 152 patients who remained comatose state after post-CA care for OHCA, 14 patients who underwent ECMO (13, extracorporeal CPR; 1, venovenous ECMO for an acute respiratory distress syndrome), and 8 patients who did not receive at least two prognostic tests (6 refused additional prognostic tests due to medical cost; 2 died within 24 h from ROSC) were excluded from this study (Fig. 1).Of the 130 patients included, 62 (47.7%) and 68 (52.3%) showed good and poor neurological outcome, respectively.The distribution of CPC scores and cause of death after post-CA care are shown in Fig. 1.
The demographic and baseline characteristics of the study population according to the primary outcome are shown in Table 1.The number of patients who underwent each of the prognostic tests was 130 (100%) PR/CR, 72 (55.4%)EEG, 127 (97.7%) serum NSE, 74 (56.9%)CT, and 116 (89.2%)MRI.The times to perform EEG, CT, and MRI were not significantly different between patients with the good and poor neurological outcomes (Table 1).

Prognostic performance of a single predictor
The prognostic values of each individual prognostic test are shown in Table 2.All poor outcome predictorsabsent PR/CR, highly malignant EEG, high NSE, low GWR-BG, and diffuse HSI on DWI-were significantly associated with poor neurological outcome (Table 2).The AUC values, sensitivities, and FPRs of each individual   3).Additionally, absent PR/CR and diffuse HSI on DWI did not produce 0% FPR (Table 3).

Prognostic performance of combining strategies of multiple predictors
The prognostic performance of combining strategies of two or more poor outcome predictors is shown in Fig. 2. All combination strategies produced 0% FPR for predicting poor neurological outcomes.Of the 18 combination strategies using multiple poor outcome predictors (the combination of two image studies was not performed), 3, 14, and 1 strategy(s) demonstrated poor (< 70%), moderate (70-90%), and excellent (> 90%) sensitivities, respectively, at 0% FPR (Fig. 2).All three combination strategies with poor sensitivity were observed in the combination strategies with two predictors (absent PR/CR + high NSE, absent PR/CR + diffuse HSI on DWI, and highly malignant EEG + diffuse HSI on DWI; Fig. 2), and the best prognostic performance was observed in the absence of PR/CR, high NSE, and diffuse HSI on DWI (91.2%, 95% CI 80.7-97.1,Fig. 2).The combination strategy of all available predictors (4 predictors, the others with one image study) rarely improved the prognostic performance (with CT: 92.8%, 95% CI 64.2-94.2;with DWI: 87.8%, 95% CI 73.8-95.9)compared with those in the combination strategies of three predictors (Fig. 2).
Imaging studies (CT and MRI) were used to verify the prognostic performance of the combination strategies, and showed no significant differences between the derivation and validation cohorts (Table 4).

Inter-rater reliability of EEG classification and imaging studies
The inter-rater reliabilities revealed moderate (κ = 0.489) and almost perfect (κ = 0.919) agreements in the interpretations of EEG and DWI, respectively (Table 5).GWR-BG showed a substantial agreement between the two reviewers (ICC, 0.808; Table 5).

Discussion
This study found that the prognostic performance of combining prognostic strategies, regardless of included predictors, not only produced 0% FPR in combination but also generally revealed improved sensitivity compared with that of predictors that show poor sensitivity (< 70%) or unacceptable FPR individually.These findings emphasize the importance of multimodal outcome prediction to guarantee a low rate of falsely pessimistic predictions, potentially leading to an inappropriate WLST in patients undergoing post-CA care 5 .Nonetheless, the sensitivities of combinations of two predictors with 0% FPR were heterogenous from poor to excellent (> 90%).Combining 3 or 4 predictors demonstrated moderate to excellent sensitivities of 78.6% or more.Interpretations of poor outcomes from both imaging studies showed numerically higher inter-rater agreement compared with those of EEG (moderate in EEG vs. substantial to almost perfect in imaging studies).www.nature.com/scientificreports/Several previous studies reported that false positive findings occurred with all the single prognostic tests currently used for prognostication, emphasizing the importance of a multimodal approach [22][23][24] , and thus, combining predictors aims to increase the sensitivity of outcome prediction and reduce the risk of false, pessimistic prognostication.We found that the prognostic performance of a single prognostic test, of which two tests failed to achieve 0% FPR.The others not only had relatively low sensitivities (< 70%), but also a wide range of 95% CIs, again emphasizing the importance of a multimodal approach.We hypothesized that adding more predictors (i.e., maximal four predictors in this study due to limited medical resources and avoiding the combination of two imaging studies) to a combination strategy would be associated with improved accuracy in predicting poor neurological outcome.However, the strategies combining four predictors, regardless of the imaging study, did not show a significant improvement in sensitivity with 0% FPR compared with that of combination strategies using three predictors, which showed the best prognostic performance.Although the heterogeneous sample size for each predictor and the retrospective nature of this registry-based study are limitations, our findings indicate that imprudently adding predictors to the combined strategy without careful consideration does not guarantee the improvement of prognostic performance and may be inefficient when considering limited medical resources and costs.In line with this suggestion, a recent external validation study of the 2021 ERC/ESICM prognostic algorithm reported that the combination of prognostic tests representing complementary and/or duplicated pathophysiology, highly malignant EEG, and absent somatosensory evoked potentials (SSEP) response, or poor CT and poor MRI revealed even lower sensitivities compared with those of each of the individual predictors alone 18 .This supports that adding predictors, such as duplicated pathophysiology with another predictor, may increase redundancy rather than sensitivity 8 .Particularly in a resource-limited setting, utilizing and combining a few predictors should be maximized while still providing high accuracy for prognostication.
From our finding, an excellent sensitivity (> 90%) was observed solely in three predictors: absent PR/CR, high NSE, and diffuse HSI on DWI.The upper boundary of the 95% CI of the FPR was 6.2%, which is close to 5% and suggests a sufficient condition for the most robust predictors 25 .Additionally, internal validation tests showed that there were non-significant differences in the prognostic performance between each cohort (derivation vs. validation).These modalities represent individually different pathophysiologic mechanisms in hypoxic-ischemic brain injury 10,26,27 , these prognostic tests were measured or performed without major confounders.Notably, MRI was performed during the homogenous phase (i.e., post TTM period, 72 to 96 h after ROSC), whereas the EEG was not (i.e., between normothermia and post TTM period).Previous studies suggested that different prognostic tests all have optimal predictive value at specific time points after the arrest, requiring exact timing and organization of the prognostic diagnostic process to avoid suboptimal sensitivity and specificity of an individual test, and post-CA care should be taken to optimize the timing of the individual prognostic parameters to ensure optimal sensitivity and specificity 8,28 .However, a prospective multicenter evaluation or an external validation test with a large sample size in an unbiased and reproducible setting would allow the generalization of the combination strategy using the three predictors proposed in this study.Given the lack of evidence for this combination strategy, we suggest that these three predictors obtained in the post-TTM period likely have the potential to improve the prognostic performance by combining them.
Among the prognostic tests in the algorithm, five imaging studies can assist in prognostication after CA by visualizing injury patterns 29 .However, there is a dearth of data for imaging studies, particularly MRI studies, that have been used in studies on the multimodal approach to prognostication.For example, a study with large cohort of 585 patients enrolled only 35 (6.0%) patients' MRI data 22 .In addition, there are no standardized recommendations as to definite times to perform, measurement techniques, and abnormal findings.Imaging methods, particularly MRI, have limitations for application such as the difficulty of moving patients out of the intensive care unit or a relatively long scan time 30 .Thus, it is reasonable to reserve the use of imaging studies for prognostication only in centers where specific experience is available 5 .There is still a lack of evidence for the prognostic value of imaging studies, and thus the current guideline does not propose specific criteria for "poor outcome likely" in imaging studies 5 .Nevertheless, imaging studies are not prone to interference from sedative drugs, and they can be assessed blindly 5 .Our finding that diffuse HSI on DWI did not yield 0% FPR as a single predictor follows prior reports 31,32 .Therefore, we suggest that DWI should be a complementary tool with other predictors, possibly for identifying different subtypes of hypoxic-ischemic brain injury after CA.The lowest sensitivity was observed for the combination of highly malignant EEG and diffuse HSI on DWI in this study.Additionally, the median time for EEG recording in this study was almost 100 h after ROSC during the heterogeneous phase (i.e., between normothermia and post-TTM period).Although this is in line with the current guideline recommendations for the timing of EEG 5 , it is still insufficient to exclude bias.EEG, which is widely available, is the most used prognostic tool after CA 33 .Several studies have demonstrated that early EEG (i.e., performed within 12 h after ROSC) has a significant prognostic performance for either good or poor neurological outcome 34,35 .Unfortunately, this study did not demonstrate the combination strategy used early EEG finding.The complementary role between EEG and MRI in comatose survivors of CA was described, and the authors explained the discordance between EEG and MRI by a different pathophysiological mechanism for their abnormal findings 36,37 .EEG is mainly sensitive to cortical brain damage, whereas MRI allows for easier identification of structural abnormalities of the neocortex, deep gray nuclei, or hippocampi in the cortical and subcortical gray matter 10 .Several studies found that patients with malignant EEG patterns do not reliably demonstrate MRI evidence of anatomic injury and suggested that performing MRI in patients with highly malignant EEG is unlikely to yield additional useful information for prognostication [36][37][38] .Based on those previous reports, our finding suggests that the low sensitivity of the combination of a highly malignant EEG and diffuse HSI on DWI supports a complementary role between EEG and MRI.Moreover, the combination strategy of using early EEG and MRI may be useful to know the prognosis of patients with CA.
The present study has several limitations to generalize for clinical practice.First, a major limitation was that all predictors suggested from the current guideline were not included for all patients (i.e., SSEP and status myoclonus were not included in this comparative analysis due to resource limitations), and this has limited the possibility to test different combinations to determine the optimal prognostic algorithm in this setting.Therefore, our study does not fully assess and compare the combination of prognostic strategies in the current ERC/ESICM prognostication strategy algorithm, which can lead to selection bias in this study 22 .Second, this was a singlecenter retrospective study with a relatively small sample size.This makes the validation test low quality and with limited generalization.Therefore, further multicenter prospective studies with larger sample sizes are required to enhance the generalizability of these findings.Third, although the prognostic tests,-PR/CR, NSE, and imaging studies-were performed or obtained in the homogeneous phase (i.e., post TTM period), the time to EEG was not well controlled since the indication for EEG was made at the discretion of the responsible physicians at various time points in this study.Fourth, the sedatives used in this study, especially midazolam, were administered during TTM and may have confounded the clinical examinations to assess PR/CR and potentially affect the EEG pattern.In this study, the examination for PR/CR was performed after TTM and without sedation; however, potential bias can produce a confounding effect from administered sedatives, which may affect these predictors (i.e., PR/CR and EEG).However, the most last (latest) EEG data, collected over 120 h after ROSC, were used in this analysis.Given the prolonged time for the prognostic test and the rarely performed WLST in this study, we suggest that this concern could be reduced despite the lack of evidence.Finally, we used a simple qualitative analysis of DWI, which may not be generally accepted.The quantitative analysis for MRI was suggested based on several studies [39][40][41] .However, it is rarely performed in clinical practice and is difficult to reproduce.Therefore, we suggest that a simple method using the extent of diffuse HSI on DWI could be more useful compared with previous complex methods when added as one predictor as part of a multimodal approach.

Conclusion
The best prognostic performance was observed in the strategy combining three predictors: absent PR/CR, high NSE, and diffuse HSI on DWI.However, the combination of four predictors did not lead to improved prognostic performance.Therefore, thoughtlessly adding tests to a combination strategy may not guarantee the improvement of prognostic performance and may be inefficient, especially when considering limited medical resources and costs.

Figure 2 .
Figure 2. Venn diagrams showing the sensitivity (upper), FPR (below), and 95% CI of each combination strategy using 2 to 4 predictors: PR/CR, EEG, NSE, and/or (a) CT; (b) DWI.Asterisk: the areas is not correlated with the prognostic performance or the number of data.CI confidence interval, CT computed tomography, DWI diffusion-weighted imaging, EEG electroencephalography, FPR false-positive rate, NSE neuron-specific enolase, PR/CR pupil light and corneal reflexes.

Table 2 .
Associations between each prognostic tests and neurological outcomes.Data are presented as n (%) or median (interquartile range).p-values < 0.05 are marked bold.PR/CR pupillary light and corneal reflexes, EEG electroencephalography, NSE neuron-specific enolase, CT computed tomography, GWR-BG gray and white matter ratio in basal ganglia level, MRI magnetic resonance imaging, HSI high-signal intensity.

Table 3 .
Prognostic performances of single predictors for poor neurological outcome.Poor neurological outcome was defined as Cerebral Performance Category score of 3 to 5. CI confidence interval, TP true positive, FP false positive, TN true negative, FN false negative, PR/CR pupillary light and corneal reflexes, EEG electroencephalography, NSE neuron-specific enolase, GWR-BG gray-white matter ratio at basal ganglia level, DWI diffusion-weighted imaging, HSI high signal intensity.

Table 4 .
Internal validation of prognostic performance for the combination strategies using imaging studies.AUC area under the curve, CI confidence interval, NPV negative predictive value, PPV positive predictive value, CT computed tomography, PR/CR pupillary light and corneal reflexes, EEG electroencephalography, NSE neuron-specific enolase, MRI magnetic resonance imaging.a p-value was calculated for the statistical comparison of prognostic performance between derivation and validation cohort by using DeLong's test.Vol.:(0123456789) Scientific Reports | (2023) 13:21880 | https://doi.org/10.1038/s41598-023-49345-1

Table 5 .
Inter-rater reliability analysis of interpretations for prognostic tests between two experts.ICC intraclass correlation coefficient, EEG electroencephalography, CT computed tomography, GWR-BG graywhite matter ratio at basal ganglia level, SD standard deviation, DWI diffusion weighted imaging, HSI highsignal intensity.